Automatic Real Time Localization of Frowning and Smiling Faces under Controlled Head Rotations

The aim of the present study was to develop a new method for fast localization of the face from streaming colour video. In the method, the face is located by analysing local properties of four facial landmarks, namely, regions of eyes, nose, and mouth. The method consists of three stages. First, the face-like skin-coloured image region is segmented from the background and transformed into the grey scale representation. Second, the cropped image is convolved with Sobel operator in order to extract local oriented edges at 16 orientations. The extracted edges are further grouped to form regions of interest representing candidates for facial landmarks. The orientation portraits, which define the distribution of local oriented edges inside the located region, are matched against the edge orientation model to verify the existence of the landmark in the image. The located landmarks are spatially arranged into the face–like constellations. Finally, the best face-like constellation of the landmark candidates is defined by a new scoring function. The test results showed that the proposed method located neutral, frowning, and smiling faces with high rates in real time from facial images under controlled head rotation variations. Key-Words: Face localization, Facial landmarks, Sobel edge detection, Frontal-view geometrical face model, Facial expressions, Head rotations